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FACTORS IN PERFORMANCE ON THE 
LAW SCHOOL ADMISSION TEST 



i 



EXECUTIVE SUMMARY 

Typically, a test— such as the Law School Admission Test (LSAT), the primary focus 
of this study— employs more than one type of item and/or different kinds of 
content for a given item type. For these multi-faceted measures, the extent to 
which the different types of test questions tap different aspects of a particular 
ability, or in fact tap several different abilities, needs to be assessed. In 
the case of the LSAT, as presently constituted, this issue does not appear to 
have been resolved. As described in Section 1 of the main report, this study was 
undertaken to clarify the internal structure of the LSAT, and shed light on the 
nature of the ability or abilities measured by the three types of test items that 
make up the LSAT-that is, reading comprehension, logical reasoning, and 
analytical reasoning. 

The study drew on data for two different forms of the LSAT, namely, the June 1991 
form and the October 1991 form. For broader perspective, the study also used 
data from the same two LSAT administrations, for a subsample composed of LSAT 
examinees identified through file -matching procedures as having taken the 
Graduate Record Examinations (GRE) General Test, between October 1988 and 
December 1991, inclusive. Time interval between GRE and LSAT testing occasions, 
without regard to order, ranged from five days to 36 months. 

Items of the same types as those used in the current version of the LSAT have 
been included in all editions of the GRE General Test since October 1981. Thus, 
it was possible to draw on the substantial body of evidence generated in the GRE 
context regarding relationships among these item types. This research is 
reviewed in Section 2 of the report. Section 2 also includes information 
pointing up (a) strong similarities in the "surface characteristics" of the three 
item types as reflected in descriptions of the three item types in testing 
program publications, and illustrative items from the LSAT and the GRE, as well 
as (b) differences between the LSAT and the GRE with respect to internal 
organization of test items, number of sections, and so on. 

Within the joint LSAT/GRE sample it was possible to 

(a) conduct parallel vithin-test analyses of correlations among the 
three item types, 

(b) assess time-related attenuating effects in patterns and levels of 
betveen-test correlations involving scaled scores and specially 
computed item- type subscores for the three common item types, and 
ultimately, by using combined data from both the LSAT and the GRE, 

(c) assess the extent to which patterns of correlations involving 
parcels of items of three types common to both tests in the combined 
LSAT/GRE sample were similar to those identified in the two separate 
within- test analyses. 
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A series of related analyses, was undertaken. 

First, separate analyses were made of within- test correlations involving 
primarily scores on item- type parcels (sets of four to six items of the same 
type) using 

(a) LSAT data for a general sample of LSAT examinees and for the 

selected sample, alluded to above, of LSAT examinees who also took 
the GRE General Test (see Section 3 ) . and 

(b) GRE data for the selected LSAT/GRE sample, as reported in Section 4 . 

A unique feature of the study was the method of pooling different items of the 
same type and position across multiple forms to create parcels used to generate 
correlations for analysis. 

Findings were generally parallel for the separate exploratory within- test factor 
analyses based on parcels of items of the three types common to both tests. In 
each analysis, when two factors were extracted, LSAT logical reasoning and 
reading comprehension items defined one factor, while LSAT analytical reasoning 
parcels defined the other. In the case of the LSAT, findings for the selected 
sample who took the GRE were in all essential respects similar to those for the 
general sample. 

These parallel within- test findings suggested in both tests these item types 
measure psychometrically distinguishable aspects of reasoning ability: aspects 

of general or informal reasoning, defined by reading comprehension items and 
logical reasoning items, on the one hand, and aspects of formal, deductive 
reasoning, defined by analytical reasoning items, on the other. 

Next, analyses were made of between- test correlations involving reported, scaled 
scores and specially computed item-type section scores. These analyses 
(described in detail in Section 5 ) were designed in part to assess effects 
associated with the fact that the LSAT observations and the GRE observations were 
collected on different testing occasions separated by intervals ranging from less 
than 10 days to 36 months. The between-test analyses included assessment of 
time -related effects on between-test correlations involving the three item types 
common to both tests. 

In these analyses, profiles of correlations involving LSAT item types and their 
GRE counterparts, computed for shorter -interval (between tests) and longer- 
interval subgroups (< 10 days versus 19—36 months) were found to be strikingly 
similar with respect to pattern. They differed only with respect to level. 

Results of the analysis of between-test correlations— observed and corrected for 
attenuation due to the presence of measurement error— involving item types common 
to both tests were consistent with the findings of the separate within-test 
factor analyses. In both instances findings suggested psychometrically 
distinguishable differences between aspects of general or informal reasoning 
measured by reading comprehension and logical reasoning item types, on the one 
hand, and aspects of formal, deductive reasoning tapped by the analytical 
reasoning items, on the other. 
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Last in the series of analyses, as described in Section 6 . intercorrelations 
involving the combined set of LSAT and GRE logical reasoning, reading 
comprehension, and analytical reasoning item- type parcels were analyzed. The 
sets of LSAT and GRE parcels used in the two separate analyses were combined to 
produce intercorrelations for these analyses. The eigenvalues suggested that two 
factors, one defined primarily by LSAT and GRE logical reasoning and reading 
comprehension parcels and the other by LSAT and GRE analytical reasoning parcels, 
sufficiently characterized the correlational structure for the combined parcels. 



Some Implications 

In essence, the study findings suggest a common underlying structure for logical 
reasoning, reading comprehension, and analytical reasoning item type regardless 
of the test (LSAT or GRE) in which they are used. The structure appears to 
involve two dimensions. One dimension is represented by logical reasoning and 
reading comprehension item types measuring general reasoning skills that appear 
to be associated with the analysis of extended discourse. The other dimension 
represents a more narrowly constrained, formal -deductive aspect of reasoning, 
measured by the analytical reasoning item type. 

For the LSAT, which currently reports only a single score to summarize 
performance involving three different item types, perhaps the most central 
conclusion supported by the findings is that 

• the logical reasoning, reading comprehension, and analytical reasoning 
item types included in the LSAT, have potential to generate more 
information than is now being conveyed by the single LSAT scaled 
score. 

That potential is suggested by the finding that the - LSAT item types measure 
psychometrically distinguishable aspects of reasoning ability. This raises the 
attendant possibility that the information provided by item- type subscores might 
prove to be useful for predictive or diagnostic purposes in the LSAT context. 

Questions concerning differential and/or incremental validity of subscores that 
might be computed are of immediate interest. For example, one score based on 
logical reasoning and reading comprehension items and a second score based on the 
analytical reasoning items would be consistent with the basic two-factor 
outcomes. Would the use of two scores, or perhaps a score for each LSAT item 
type, result in improved prediction of first-year law school grades generally, 
grades in particular courses or clusters of courses, grades in second-year 
courses? 

A study of the comparative validity of LSAT subscores such as those noted above, 
for predicting such criteria— in general samples, and in samples defined by ethnic 
group membership, age, gender, undergraduate major, and so on— would contribute 
toward resolution of academically, psychometrically, and socially important 
"differential validity" questions in the current LSAT context. 
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Closely related to the foregoing are questions concerning incremental validity. 
For example, does differential weighting of item- type subscores result in better 
prediction of pertinent criteria (for example, grades in successive years of 
legal education), than is provided by a total scaled score based on a simple sum 
of section scores, in general samples of law students? In subgroups defined by 
ethnicity, gender, age, undergraduate major, and so on? 

Study findings indicate that between- test correlations involving LSAT reading 
comprehension, logical reasoning, and analytical reasoning subscores and 
corresponding GRE subscores are differentially resistant to time-related 
attenuating influences. These findings suggest the possibility of differences 
in relative stability for the abilities involved. In the present study, 
inferences about relative stability, of course, are based on between- test 
correlations for the respective item- type subscores. It seems important to make 
a direct assessment of test-retest stability, short- and long-term, for LSAT 
item- type subscores. 

These and other study findings have incidental implications for the GRE context. 
For example, the findings tend to confirm and extend conclusions based on GRE 
studies, namely, that logical reasoning items and analytical reasoning items are 
measuring psychometrically distinguishable aspects of reasoning ability. Thus, 
research questions such as those raised above for the LSAT, also have 
implications for continued research in the GRE context. 

That LSAT logical reasoning, reading comprehension, and analytical reasoning 
items and their GRE counterparts have a common factor structure is important 
because this finding suggests that future research involving these item types in 
the LSAT context can draw on relatively extensive GRE research findings (such as 
those summarized briefly in Section 2 of the main report) both for formulating 
working hypotheses and for evaluating LSAT research outcomes. It also follows 
that as LSAT research findings involving these item’ types accrue, the LSAT 
findings in turn can usefully inform research in the GRE context. 

Jointly planned research projects involving item types common to both tests might 
expedite attainment of objectives common to both testing programs— for example, 
clarifying distinctions between logical reasoning and reading comprehension. 

In this connection, given the observed affinity between logical reasoning and 
reading comprehension— combined with hints of distinctiveness— it is noteworthy 
that the version of the logical reasoning item type considered in this study for 
both the LSAT and the GRE involves heavy reading comprehension requirements. 

Accordingly, logical reasoning and reading comprehension are "linked” to some 
degree by heavy reading demands. To the extent that it is possible to measure 
"logical reasoning" using item types with limited reading demands, progress may 
be made in clarifying distinctions between "logical reasoning" and "reading 
comprehension," by cooperative research projects involving experimental logical 
reasoning and analytical reasoning items, and operational items from both tests, 
along lines followed in GRE research conducted by Emmerich, Enright, Rock, and 
Tucker (1991), for example. 
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In reaching decisions regarding score definition and score reporting, the results 
of this study involving data from both tests, suggest that both testing programs 
might benefit from research projects capitalizing on the common structure that 
appears to underly performance on the three types of items that are common to 
both the LSAT and the GRE General Test. 
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ABSTRACT 

This study was undertaken to clarify the internal structure of the Law School 
Admission Test (LSAT) , and shed light on the ability or abilities measured by the 
three item types that make up the test (logical reasoning, analytical reasoning, 
and reading comprehension) . The study used data for two forms of the LSAT in (a) 
general samples of LSAT examinees, and (b) a subsample of LSAT examinees 
identified as having also taken the GRE General Test (GRE) , which includes the 
three item types that make up the LSAT. 

Within the joint LSAT/GRE sample, it was possible to conduct separate, parallel 
factor analyses of parcels of item types common to both tests, assess time- 
related effects on between- test correlations, and analyze factor structure 
underlying performance on combined LSAT and GRE parcels of logical reasoning, 
analytical reasoning, and reading comprehension items. Unique features of the 
study were (a) the use of data from both the LSAT and the GRE General Test to 
assess structure in item types common to both tests, and (b) the method of 
pooling different items of the same type and position across multiple test forms 
in order to create parcels used to generate correlations for analysis. 

Study findings suggested a similar structure for logical reasoning, analytical 
reasoning, and reading comprehension item types, regardless of the test (LSAT or 
GRE) in which they are used. The structure entails two dimens ions-one involving 
aspects of general reasoning as measured by logical reasoning and reading 
comprehension item types, and the other more narrowly constrained, formal- 
deductive aspects of reasoning tapped by the analytical reasoning item type. 

Implications of the findings are discussed. 
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FACTORS IN PERFORMANCE 
ON THE LAW SCHOOL ADMISSION TEST 



i 



SECTION 1. INTRODUCTION 



The Law School Admission Test (LSAT) is described (Law School Admission Services, 
1991, p, 2) as being designed 

. to measure skills that are essential for success in law school: 
the ability to read and comprehend complex texts with accuracy and 
insight, organize and manage information and draw reasonable inferences 
from it, reason critically, and analyze and evaluate the reasoning and 
argument of others. These skills are typically developed over a long 
period of time. The LSAT provides candidates and law schools with a 
useful measure of a candidate’s ability to handle new information, using 
well-developed reading skills and careful and logical thought as the 
principal tools . " 

The skills alluded to above are assessed operationally by three different types 
of multiple-choice test questions or items, labelled, reading comprehension, 
analytical reasoning, and logical reasoning, described (for example, LSAT, 1991), 
as follows: 

• Reading comprehension questions require test takers to read carefully 
and accurately, to determine the relationships among the various parts 
of the passage, and to draw reasonable inferences from the material in 
the passage, 

• Each logical reasoning question requires the examinee to read and 
comprehend the argument or the reasoning contained in a short passage, 
and then answer one or two questions about it. ,The questions test a 
variety of abilities involved in reasoning logically and critically 
(including, for example,) drawing reasonable conclusions from given 
evidence or premises. 

• Analytical reasoning items are designed to measure the ability to 
understand a structure of relationships and to draw conclusions about 
the structure. The examinee is asked to make deductions from a set of 
statements, rules, or conditions that describe relationships among 
entities such as persons, places, things, or events. 

These three item types have been used in all forms of the LSAT developed since 
1982, and are treated operationally as measuring related aspects of one general 
underlying ability. For example, test performance is summarized by a single, 
LSAT scaled score based on the total number of correct responses to test items, 
without regard to type. 
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Purpose of the Present Study 



If a test employs more than one type of item and/or different kinds of content 
for a given item type, the extent to which the different types of test questions 
tap different aspects of a particular ability, or in fact tap several different 
abilities, needs to be assessed. In the case of the LSAT as presently 
constituted, this issue does not appear to have been resolved. 1 

The present study was undertaken to clarify the internal structure of the LSAT, 
and shed light on the ability or abilities measured by LSAT reading 
comprehension, logical reasoning, and analytical reasoning item types. The study 
drew on data from two regularly scheduled LSAT administrations, namely, the June 
1991, and October 1991 administrations. 

For broader perspective, the study also draws on data for a subsample from the 
same two LSAT administrations, composed of LSAT examinees identified as having 
also taken the GRE General Test (for example, ETS , 1990) between October 1988 and 
December 1991. Using data for this selected sample it was possible to conduct 
parallel within-test analyses involving item types common to both tests, examine 
between-test correlations involving scaled scores and specially computed item- 
type subscores, and ultimately to identify factors underlying performance on 
parcels of items of three types that are common to both tests, using combined 
LSAT and GRE data. 

The study also drew on the substantial body of evidence regarding relationships 
among these item types based on research in the GRE context (to be reviewed 
later) involving GRE reading comprehension, logical reasoning, and analytical 
reasoning items similar to those used in the LSAT. These three item types have 
been included in all editions of the GRE since October 1981. 

Detailed descriptions of study data and the analytical procedures used to attain 
study objectives, as well as related findings, are provided in the remaining 
sections of this report. A general overview of the organization of the report 
and the general analytical approach employed in the study, is provided below. 



1 A review of Volumes I-IV of Reports of LSAC Sponsored Research (LSAC, 1976a, 1976b, 1977, 1984) 
reveals only one study (Carlson, 1976b) — authorized by the LSAC In 1966 and reported In 1970 — concerned 
with Identifying the factor structure of an extant version of the LSAT In order to answer the question: 
What are the abilities being measured by LSAT Items? According to Carlson (p. 12), "It was hoped that 
a class If lcatlon of Items based on abilities would provide a useful supplement or alternative to the 
present class If lcatlon based on Item type . . Carlson factored matrices of Interitem coefficients 
for Items of the eight different types Included In the 1966 version of the LSAT, and Identified seven 
factors. The first factor, called "verbal ability," was defined primarily by reading comprehension 
Items; the second factor, called "verbal Inductive reasoning," was defined primarily by a subset of 
"Principles and Cases" Items." The "Principles and Cases" Item type used until June 1982, was ". . . 
developed to test the candidate's ability to reason logically" (McPeek, Pitcher, and Carlson, 1976b, 
emphasis added). During the course of the present study, another LSAC-sponsored factor analytic study 
(Camllll, Wang, and Fesq, 1992) Involving several forms of the LSAT used between June 1989 and October 
1990 --the same version of the LSAT as that under consideration herein — was reported. Findings reported 
herein for LSAT forms used In the June 1991 and October 1991 admlnlstrat Ions are consistent with those 
reported by Camllll, Wang, and Fesq (1992) for the forms used earlier. 
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Overview 



Section 2 provides (a) a detailed review of previous research findings regarding 
the internal structure of the GRE General Test, and the nature of its constituent 
item types, especially GRE logical reasoning, reading comprehension, and 
analytical reasoning, and (b) a brief comparison of the composition of both the 
LSAT and the GRE. 



Analyses of the Internal Structure of the LSAT 

Section 3 describes analyses of the internal structure of the LSAT— that is, its 
dimensionality, the principal concern of the present study. Dimensionality was 
assessed by analyzing intercorrelations of subscores based on LSAT reading 
comprehension, logical reasoning, and analytical reasoning items, and smaller 
subsets (parcels) of LSAT items within each item type. For example, exploratory 
factor analyses were conducted using data for (a) general LSAT samples taking 
different forms of the LSAT used in the June 1991 and October 1991 test 
administrations, and (b) the selected subsample from those two test 
administrations for whom GRE data were also available. 

Analyses involving LSAT performance for the selected subsample of LSAT examinees 
(those who also took the GRE) yielded results that were consistent with the 
outcomes observed for the general LSAT samples. The similarity in outcomes 
suggested, among other things, that data for the selected LSAT sample also 
provide a sound basis for drawing inferences about LSAT dimensionality. 



Assessing GRE Dimensionality in the Selected LSAT/GRE Sample 

Section 4 describes (a) an analysis of factors underlying performance on a full 
array of GRE item-type parcels, designed in part to shed light on the extent to 
which the GRE data adequately represent the GRE test taking population, and (b) 
an analysis of factors underlying performance on parcels of items of the three 
types common to both the LSAT and the GRE. 

The possibility of unrepresentative findings due to selected samples is a 
pertinent consideration generally. Questions regarding representativeness are 
especially germane here because the members of the selected LSAT/GRE sample took 
some 21 different generic forms of the GRE . on over 30 different testing 
occasions spanning a total of 42 months. 

GRE data for a general (unselected) sample of GRE examinees were not included in 
the study. Thus it was not possible to assess directly the issue of 
representativeness of findings based on the selected GRE data set, as was done 
in the case of the LSAT data. Accordingly, the issue was addressed indirectly 
by evaluating the results of exploratory analyses designed to identify factors 
underlying performance on GRE items of all nine types, in the selected sample, 
in light of previous research. These analyses were based on pooled within-form 
correlations in data aggregated across 21 generic forms of the GRE. 
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Results of the exploratory analyses, involving a full array of GRE item-type 
parcels, conformed generally to previously reported patterns, especially that 
identified by Emmerich, Enright, Rock, and Tucker (1991) in research involving 
both operational and experimental logical reasoning and analytical reasoning GRE 
item types. The orderly nature of GRE findings in the selected sample indicated 
that data for this sample provided a sound basis for drawing inferences regarding 
GRE dimensionality generally, as well as the dimensionality of the three item 
types that are common to both the LSAT and the GRE. 

Based on the foregoing, analyses of factor structure involving parcels of GRE 
reading comprehension, logical reasoning, and analytical reasoning items were 
conducted. These analyses parallel those involving the three LSAT counterparts, 
alluded to above, and the findings of the two within-test analyses also were 
generally parallel. 

Section 4 also provides information regarding the characteristics of the joint 
LSAT/GRE sample, the nature of the GRE data involved and the procedures that were 
used in aggregating data across forms and testing occasions, as well as the 
factor analyses outlined above. 



Evaluation of Between-Test Correlations 

The correlational findings reported in Section 3, involving LSAT data only, and 
Section 4, involving GRE data only, reflect concurrent, within-test relationships 
involving parcels of test items-including, in the case of the GRE, sets of 
concurrent, within-test observations based on different forms of the test, taken 
on 37 separate testing occasions. 

Among other things, these findings suggested that outcomes of correlational 
analyses involving GRE data only, as well as those involving LSAT data only, were 
not unduly influenced by (potentially interesting and probably complexly 
interacting) variables associated with selection into the LSAT/GRE sample. 

However, as described in detail in Section 5 . in all instances, the LSAT and the 
GRE were taken on separate testing occasions under different testing conditions. 
The time interval between the two tests ranged from less than 10 days to 
approximately 36 months, and the order of test taking was not the same for all 
members of the joint sample. 

Thus, correlations between LSAT and GRE scores (total scores, item scores, parcel 
scores, and so on) may be affected to some extent by factors such as the 
following: 

(a) conditions affecting performance on one testing occasion but not the 
other (anxiety, luck, illness, preparation or lack of preparation, 
shifts in motivation, and so on), 

(b) differential change (growth or decline) in the abilities being 
tapped by both tests , due to individual differences in experiences 
(over periods of up to three years between tests), 
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(c) "short-term practice effects," due to proximity of test dates; and 
so on. 

Analyses of between-test correlations reported in Section 5 . revealed time- 
related trends in correlations involving both scaled total scores and specially 
computed subscores for reading comprehension, logical reasoning, and analytical 
reasoning items. For example, observed between-test relationships were stronger 
in data for subgroups taking the two tests within 10 days, than in data for 
subgroups in which time between testing occasions was 19-36 months. In addition, 
when classified according to time interval between test occasions and sequence 
of testing, subgroups differed significantly with respect to average test 
performance— true for both the LSAT and the GRE. 

The between-test correlational findings involving item-type subscores from the 
LSAT and the GRE suggested the possibility of differences in relative stability 
over time of the abilities tapped by the three item types— for example, the 
possibility of differential change in the abilities being measured. Explicating 
the observed time-related trends and/or evaluating the relative influence of 
particular variables that may have affected observed relationships between LSAT 
and GRE scores is outside the scope of the present study. 

An evaluation is made of both observed correlations between LSAT variables 
(scaled scores, item-type section scores, and so on) and GRE variables, and 
correlations after correction for attentuation due to measurement error. 



Exploratory Factor Analyses Involving Parcels of Item Types Common to Both Tests 

Apart from their substantive implications, the findings reported in Section 5 
indicate clearly that the interpretation of between-test relationships is 
complicated by the fact that performance on the two sets of test items was 
observed at different times and under different conditions. 

Factor analyses involving the combined parcels of items of the three types that 
are common to both the LSAT and the GRE— that is, the parcels that were used in 
the separate within-test analyses— are reported in Section 6 . Despite the 
interpretive complications alluded to above, the analyses shed additional light 
regarding the classification, according to ability domain, of reading 
comprehension, logical reasoning, and analytical reasoning item types for both 
the LSAT and the GRE. 

Findings of the analysis of combined parcels of the three item types were 
generally similar to the findings of parallel within-test analyses. On balance, 
these findings suggest that a common factor structure underlys performance on the 
three types of items that are common to the LSAT and the GRE, regardless of the 
test for which they are developed. 

Section 7 summarizes study findings and suggests some implications for both the 
LSAT and the GRE contexts . 
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SECTION 2. REVIEW OF RELATED RESEARCH IN THE GRE CONTEXT 



As indicated in Section 1, reading comprehension, logical reasoning, and 
analytical reasoning items, similar in type to the corresponding LSAT item types, 
are used in forms of the Graduate Record Examinations (GRE) General Test. In the 
GRE context, these item types are treated operationally as belonging to two 
different ability domains, namely, an analytical ability domain and a verbal 
ability domain, respectively. 

For the GRE, "analytical ability" is defined operationally by the logical 
reasoning and analytical reasoning item types, which are characterized as 
measuring somewhat different aspects of ". . . the ability to think analytically" 
(ETS, 1990, p. 41). "Verbal ability" is defined by the reading comprehension 
item type and three other verbal item types — analogy , antonym and sentence 
completion Questions— characterized collectively as measuring aspects of " . . . 
ability to reason with words in solving problems . . . (that is) the ability to 
discern, comprehend, and analyze relationships among words or groups of words and 
within larger units of discourse such as sentences and written passages (and so 
on)" (ETS, 1990, p. 31). 

Separately scaled number-right scores are reported for GRE analytical ability and 
GRE verbal ability, respectively, as well as for GRE quantitative ability. 2 

GRE "quantitative ability" is defined by quantitative comparison, regular 
mathematics, and data interpretation item types which are characterized as 
measuring . . . basic mathematical skills, understanding of elementary 
mathematical concepts, and ability to reason quantitatively and to solve problems 
in a quantitative setting" (ETS, 1990, p. 36). 

Results of factor analyses involving GRE verbal, quantitative, and analytical 
items or item parcels, summarized in Table 2.1, generally tend to support a 
three-factor hypothesis, corresponding to divisions of the test for which 
separate scores are reported. That is, the items appear to measure 
psychometrically distinguishable verbal, quantitative, and analytical 
abilities-more clearly so for items in the GRE verbal domain and the GRE 
quantitative domain, than for items in the GRE analytical ability domain. 



2 In contrast to legal education. In which the first-year curriculum tends to be relatively uniform from 
school to school, graduate education is "notoriously" diverse. This fundamental difference is reflected in the 
different approaches to admissions testing for graduate education and legal education. The GRE General test, 
which reports three scores, must be appropriate for disciplines that differ on a number of grounds, e.g., the 
extent to which they are verbally or quantitatively oriented. The LSAT, not being constrained in this way, 
summarizes performance on three different item types in a single score in order to measure a more general 
reasoning ability. 
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In the GRE verbal domain, two closely related subdomains have been identified 
relatively consistently, namely, 

• "reading comprehension," a factor typically defined by the GRE reading 
comprehension item type— calling for discourse-level analysis— alone , or in 
combination with the sentence-completion item type that involves sentence- 
level analysis, and 

• "vocabulary," a factor typically defined by analogy and antonym items, 
and (less consistently) by sentence-completion items— item types that 
do not require discourse-level analysis. 

Strieker and Rock (1985), commenting on the fact that these two verbal subdomains 
did not emerge in their study, suggested that the outcome might be due to 
differences across studies in sample selection, factor methodology, and so on. 

The GRE quantitative ability domain has been consistently and clearly defined by 
the quantitative comparison and regular mathematics item types. Data 
interpretation items typically have somewhat lower loadings than either of the 
other two item types on quantitative factors identified in these analyses, but 
nonetheless tend to be more closely identified with the quantitative ability 
domain than with either of the other GRE ability domains (see also Kingston and 
Dorans, 1982). 

However, research findings indicate that in the GRE analytical ability domain, 
the alliance between component item types is an uneasy one. For example, GRE 
analytical reasoning and GRE logical reasoning items tend to have less common 
variance than do item types within the GRE verbal and GRE quantitative ability 
domains. Furthermore, what they have in common tends to overlap substantially 
with item types represented in either the verbal or the quantitative domain (for 
example, Strieker and Rock (1985: p. 25). 

Schaeffer and Kingston (1988) found that a GRE analytical ability factor was 
defined principally by the GRE analytical reasoning item type, and concluded from 
this and other findings, as follows (p. 9): 

"This study has suggested that an analytical dimension in the GRE General 
Test may be defined by the analytical reasoning items and not by the 
logical reasoning items. This finding raises doubt about the utility of 
including analytical and logical reasoning items in the same score" 
(emphasis added). 

In an analysis of GRE factor structure in two samples (Emmerich, Enright, Rock, 
and Tucker, 1991), in which three sets of operational GRE item types were 
constrained to have nonzero loadings only on the factors which they were expected 
to define, analytical reasoning parcels had strong loadings on the "analytical 
ability" factor (loadings ranged between .83 and .87), whereas those for logical 
reasoning parcels were comparatively considerably weaker (ranging between .38 and 
.56 across analyses in two samples [Emmerich et al . , p. 38]). 

In a more general, unconstrained exploratory analysis involving both operational 
and experimental logical reasoning and analytical reasoning GRE item types, two 
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of three operational logical reasoning parcels loaded more heavily on the 
"verbal" factor than on a factor labelled "formal-deductive reasoning"-a factor 
defined primarily by very strong loadings for three operational GRE analytical 
reasoning parcels. 

Results of studies involving subscores based on GRE item types (for example, 
Wilson, 1985; 1986) also indicate that GRE logical reasoning items and GRE 
analytical reasoning items may tap somewhat different underlying abilities. 

For example, correlations between GRE logical reasoning and GRE analytical 
reasoning subscores tended to be lower than correlations between GRE logical 
reasoning and GRE verbal item- type subscores, and also lower than those between 
GRE analytical reasoning subscores and subscores on GRE quantitative item 
types . 3 

For examinees classified by major area of study, patterns of relative performance 
on GRE analytical reasoning and logical reasoning were consistent with 
expectation based on major-area differences in relative performance on 
quantitative and verbal items— that is, majors in primarily verbal fields had 
relatively higher means on GRE reading comprehension and logical reasoning 
subscores than on subscores for quantitative item types, while majors in 
quantitatively demanding fields performed relatively better on GRE quantitative 
comparison items and analytical reasoning items than on either GRE reading 
comprehension or logical reasoning items. 4 

Also, an ethnic group known to be characterized by higher means on GRE 
quantitative ability than on GRE verbal ability, namely, Asian Americans, had 
higher relative standing on GRE analytical reasoning than either GRE logical 
reasoning or GRE verbal items, without regard to field . 5 

Thus, research findings from the GRE context indicate that the logical reasoning 
item type appears to tap abilities that are closely akin to those measured by the 
reading comprehension item type, hence may belong to the verbal ability domain, 
rather than to the "analytical ability" domain. And, as suggested by Schaeffer 



3 Wightman and Muller (1990b) reported Intercorrelations of Item- type section scores for LSAT analytical 
reasoning, logical reasoning, and reading comprehension from a recent version of the LSAT as follows: analytical 
reasoning and reading (r = .49), logical reasoning and reading (r » .67), and analytical reasoning and logical 
reasoning (r ® .53). 

4 Pitcher (1977b, Table C.2, pp. 480-481) reported means for law students grouped by undergraduate major 
field, on experimental logical reasoning and quantitative comparison Items, as well as on operational data 
Interpretation and reading Items. Profiles of logical reasoning and reading means were quite similar across 
fields, as were profiles of data Interpretation and quantitative comparison means. In both cases, the profiles 
conformed to a pattern consistent with expectation based on differences among the fields involved with respect 
to verbal-relative-to-quantitative emphasis in curricular content. 

5 Means and standard deviations for LSAT section scores were reported by Wightman and Muller (1990a) for 
generally comparable ethnic subgroups. The major-field mix In the sample was not reported. However, Asian 
Americans registered the highest relative standing on AR (mean « 18.83, approximately .15 standard deviation 
units above the estimated grand mean of 17.795); z-scaled mean on LR was -.16. 
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and Kingston (1988), the "analytical ability" domain itself may be defined more 
validly by the analytical reasoning item type alone than by a combination of 
logical reasoning and analytical reasoning item types. 



Characteristics of Item Types Common to Both Tests 



Published descriptions of the abilities tapped by LSAT reading comprehension, 
logical reasoning, and analytical reasoning items (for example, LSAT, 1991), are 
very similar to those published for the GRE counterparts (for example, ETS , 
1990). Excerpts from these published descriptions, as well as illustrative 
logical reasoning, analytical reasoning, and reading comprehension items from the 
two testing contexts, are summarized in a series of exhibits (Al through C2b) . 
The exhibits are arranged to provide side-by-side descriptions of an item type 
from the two testing contexts, followed by a side-by-side display of examples of 
items of the type under consideration. 




2 ? 



Exhibit Al. Descriptions of the Logical Reading Item Type: LSAT and GRE Versions 
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Exhibit Cl. Descriptions of the Reading Comprehension Item Type: LSAT and GRE Versions 

GRE General Test LaM School Admission Test 
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• the tone of the passage or the author's attitude as It Is revealed in 
the passage. 



Exhibit C2a. Illustrative GRE Reading Comprehension Items (Abridged Options) 

Directions: Each passage in this group Is followed by questions based greater the danger of gross mistakes;. The overall scheme by which 

on Ito content. After reading a passage, choose the best answer to each genetic Information Is rationed out in an organism, therefore, must in* 

question. Answer all questions following a passage on the basis of what volve a compromise between two conflicting priorities: precision and the 

is stated or trolled In the passage. avoidance of gross mistakes. 
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Based on the strong surface similarities evident in these exhibits, it seems 
clear that although items for the two tests are developed to somewhat different 
specifications (subject matter, type of situations depicted, relative emphasis 
on particular subskills, and so on) logical reasoning, analytical reasoning, and 
reading comprehension items from the LSAT and the GRE can be thought of as 
measuring generally similar skills and abilities, and are in fact quite similar 
with respect to general characteristics, subskills targeted for assessment, and 
so on. 

Differences between the two tests in the way in which these particular item types 
are presented to examinees, the number of items included, amount of time devoted 
to each item type, and so on, are indicated in Table 2.2. 
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Table 2.2. 

General Organization of Tasks in the LSAT 
and in the GRE General Test, Respectively 



LSAT sections 



Logical Reasoning 
Reading Comprehension 
Analytical Reasoning 
Logical Reasoning 
Variable section 
TOTAL SCORE 



GRE sections 
Verbal section 1 



NO. 

items 


T ime 


24 


35 


minutes 


28 


35 


minutes 


24 


35 


minutes 


25 


35 


minutes 


Variable 

101 


35 


minutes 



38 30 minutes 



Item type 

Analogy 9 
Sentence Completion 7 
Reading Comprehension 11 
Antonym 1 1 



Verbal section 2 



38 30 minutes 



Analogy 9 

Sentence Completion 7 

Reading Comprehension 11 

Antonym 11 

Quantitative section 1 

I tern type 30 

Quantitative comparison 15 

Regular mathematics 10 

Data interpretation 5 

Quantitative section 2 30 



30 minutes 



30 minutes 



Quantitative comparison 15 



Regular mathematics 10 

Data interpretation 5 

Analytical section 1 25 

Item type 

Logical reasoning 6 

Analytical reasoning 19 

Analytical section 2 25 



30 minutes 



Item type 

Logical reasoning 
Analytical reasoning 



Number of items 

6 

19 



Variable section (not scored) Variable 30 minutes 



Total nunber of test items 186 
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In forms of the LSAT, including the forms involved in the present study, logical 
reasoning, reading comprehension, and analytical reasoning items are presented 
in four separately timed 35-minute sections, each of which is homogeneous with 
respect to item type. Two of these sections are made up of logical reasoning 
items. A fifth (variable) section is included for purposes of test development. 
Performance on the items in the variable section does not affect a candidate’s 
LSAT score. 

The ordering of LSAT sections differs from form to form, and across operational 
versions of given generic forms. 

The GRE includes a total of seven separately timed sections, each with a 30- 
minute time limit, one of which is variable. In a given version of the test, 
sections are presented in such a way as to require examinees to work sequentially 
through two different sets of tasks from the respective ability domains— for 
example, a 30-minute verbal set, a 30-minute quantitative set, a 30-minute 
analytical set, a second quantitative set, a second analytical set, a second 
verbal set, followed by a variable set. 

The two sections of GRE items that constitute a given ability measure, although 
developed to comparable specifications, are not designed to be parallel with 
respect to validity, difficulty level, and so on. 



SECTION 3. ASSESSING THE DIMENSIONALITY OF THE LSAT 



This section describes factor analyses that were undertaken 

(a) to explore the extent to which the Law School Admission 
Test— made up of logical reasoning, analytical reasoning, 
and reading comprehension item types, organized as 
indicated in Table 3 . 1-raeasures more than one identifiable 
aspect of reasoning, and 

(b) to help clarify the aspects of reasoning tapped by the three LSAT 
item types. 



Table 3.1. 



Organization of the LSAT 



LSAT Sections 


Number of Items 


Acronym 




Time 


Logical reasoning 


24 


LR24 


35 


minutes 


Reading comprehension 


28 


RC28 


35 


minutes 


Analytical reasoning 


24 


AR24 


35 


minutes 


Logical reasoning 


25 


LR25 


35 


minutes 


Variable section 


Variable 




35 


minutes 


TOTAL SCORE 


101 


LSAT 









i 

i 



Extensive research findings in the GRE context indicate that the GRE versions of 
the item types used in the LSAT are tapping psychometrically distinguishable 
aspects of reasoning. And, as we have seen, published descriptions of the three 
LSAT item types are very similar to the published descriptions of their GRE 
counterparts, and actual items of the three types from the LSAT appear to be 
quite similar, with respect to surface characteristics, to those appearing in the 
GRE General Test. 



General Analytical Rationale 

With respect to the dimensionality issue, GRE research findings indicate 
relatively clearly that 

• the aspects of reasoning measured by the logical reasoning item type 
are closely related to those tapped by the reading comprehension item 
type, but questions as to the ability-domain classification of the two 
item types have not been resolved, but 



• the aspects of reasoning tapped by the analytical reasoning item type 
appear to be psychometrically distinguishable from those tapped by 
reading comprehension items and logical reasoning items— albeit not as 
clearly so for logical reasoning as for reading comprehension item 
types . 

Judging from correlational findings involving LSAT item-type section scores, 
available for two LSAT forms involved in the present study (from unpublished ETS 
internal analyses) , patterns of relationships involving the respective item types 
in the LSAT context, suggest interpretive inferences that are similar to those 
outlined above for the GRE counterparts. The LSAT findings are summarized in 
Table 3.2. 
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Table 3.2. 

Observed Correlations and Correlations After Correction for Attenuation: 
LSAT Section Scores, June 1991 and October 1991 Forms of the LSAT* 







June 


1991 


LSAT Form 




October 


■ 1991 




Section 


LR25 


LR24 


RC28 


AR24 


LR25 


LR24 


RC28 


AR24 


LR25 


(.78) 


.97 


.91 


.71 


(.77) 


.96 


.89 


.68 


LR24 


VO 


(.79) 


.89 


.72 


.74 


(.77) 


.87 


. 64 


RC28 


.72 


.71 


(.80) 


.63 


.69 


.68 


(.79) 


.59 


AR24 


.55 


.56 


.50 


(.77) 


.52 


.49 


. 46 


(.76) 



Note . Observed correlations are shown below the diagonal; corrected correlations 

are shown above the diagonal; diagonal elements are estimated KR-20 reliabilities, 
used to correct the correlations. 

* Data from unpublished ETS internal test analyses for forms of the LSAT used in 
the present study; KR-20, internal-consistency estimates. 



The pattern of correlations shown in Table 3.2 is particularly instructive 
because corrected correlations for the two LSAT logical reasoning sections 
(r = .97 and r = . 96)— coefficients approaching 1.00, consistent with the fact 
that both sections are intended to measure the same underlying skills and 
abilities— can be contrasted directly with corrected correlations between the 
logical reasoning section scores and scores for sections involving two different 
item types, namely, 

score for the reading comprehension item type, tapping abilities 
that are quite similar to those tapped by the LSAT logical reasoning 
sections (r’s ranging between .87 and .91, but lower than those for 
the two logical reasoning sections), and 

score for the analytical reasoning item type, tapping abilities that 
do not tend to overlap markedly with those tapped by logical 
reasoning (indicated by corrected correlations ranging between .64 
and .72) or by reading comprehension, for which corrected 
correlations with analytical reasoning were .59 and .63. 

It can also be seen that correlations between LSAT reading comprehension and 
analytical reasoning (r = .59 and r = .63) are lower than those between LSAT 
logical reasoning and analytical reasoning (r's between .64 and .72). 

The marked overlap between reading comprehension and logical reasoning reflected 
in the corrected correlations in Table 3.2, suggests that two factors might 
explain the basic underlying patterns of intercorrelations, namely, one factor 
defined by logical reasoning and reading comprehension parcels, and a second 
factor defined exclusively by analytical reasoning parcels. 



(a) 



(b) 
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